 |
Observe Users
Once you have a prototype drawn or mocked up, you can begin to show it to people to get reactions to it. The fourth step, called user observation, lets you test the workability of your product design by watching and listening carefully to users as they work with your prototype. Although it is possible to collect far more elaborate data, observing users is a quick way to obtain an objective view of your product. Before you do any testing, take time to figure out what you're testing and what you're not. By limiting the scope of the test, you're more likely to get information that will help you solve a specific problem. You can use the information you gather about your target audience to help you pick participants for your user observation; find people who have the same demographic background and experience level as the typical user in your target audience. Your participants will work through one or more specific tasks. These tasks can be based on the task analyses that you performed earlier in the design process. After you determine which tasks to use, write them out as short, simple instructions. Your instructions to the participants should be clear and complete but should not explain how to do things you're trying to test. See the following section, "Ten Steps for Conducting a User Observation," for more information about how to conduct a user observation; it includes a series of sample steps on which you can base your own user observation.
During the user observation, record what you learn about your design; you'll be using this information to revise your prototype. Once you've revised your prototype, conduct a second user observation to test the workability of the changes you've made to your design. Continue this iterative process of creating prototypes and conducting user observations until you feel confident that you've fully addressed the needs of your target audience.
Ten Steps for Conducting a User Observation
The following steps provide guidelines that you can use when conducting
a simple user observation. Remember, this test is not designed as an experiment, so you will not get quantitative data that can be statistically analyzed. You can, however, see where people have difficulty using your product, and you can then use that information to improve your product.
Most of these steps include some explanatory text that contains sample statements that you can read to the participant. Feel free to modify
the statements to suit your product and the situation.
- Introduce yourself and describe the purpose of the observation (in
very general terms). Most of the time, you shouldn't mention what you'll be observing.
Set the participant at ease by stressing that you're trying to find problems in the product. For example, you could say something like this:
- "You're helping us by trying out this product in its early stages."
- "We're looking for places where the product may be difficult to use."
- "If you have trouble with some of the tasks, it's the product's fault, not yours. Don't feel bad; that's exactly what we're looking for."
- "If we can locate the trouble spots, then we can go back and improve
the product."
- "Remember, we're testing the product, not you."
- Tell the participant that it's OK to quit at any time.
Never leave this step out. Make sure you inform participants that they can quit at any time if they find themselves becoming uncomfortable. Participants shouldn't feel like they're locked into completing tasks. Say something like this:
- "Although I don't know of any reason for this to happen, if you should become uncomfortable or find this test objectionable in any way, you are free to quit at any time."
- Talk about the equipment in the room.
Explain the purpose of each piece of equipment (hardware, software, video camera, tape recorder, microphones, and so forth) and how it will be used in the test.
- Explain how to think aloud.
Ask participants to think aloud during the observation, saying what comes to mind as they work. By listening to participants think and plan, you'll
be able to examine their expectations for your product as well as their intentions and their problem-solving strategies. You'll find that listening to users as they work provides you with an enormous amount of useful information that you can get in no other way.
Some people feel awkward or self-conscious about thinking aloud. Explain why you want participants to think aloud and demonstrate how to do it. For example, you could say something like this:
- "We have found that we get a great deal of information from these informal tests if we ask people to think aloud as they work through
the exercises."
- "It may be a bit awkward at first, but it's really very easy once you get used to it. All you have to do is speak your thoughts as you work. If you forget to think aloud, I'll remind you to keep talking. Would you like me to demonstrate?"
- Explain that you will not provide help.
It is very important that you allow participants to work with your product without any interference or extra help. This is the best way to see how people really interact with the product. For example, if you see a participant begin to have difficulty and you immediately provide an answer, you will lose the most valuable information you can gain from user observation--where users have trouble and how they figure out what to do.
Of course, there may be situations in which you will have to step in and provide assistance, but you should decide what those situations will be before you begin testing. For example, you may decide that you will allow someone to struggle for at least three minutes before you provide assistance. Or you may decide that there is a distinct set of problems on which you will provide help. However, if a participant becomes very frustrated, it's better to intervene than have the participant give up completely.
As a rule of thumb, try not to give your test participants any more information than the true users of your product will have. Here are some things you can say to the participant:
- "As you're working through the exercises, I won't be able to provide help or answer questions. This is because we want to create the most realistic situation possible."
- "Even though I won't be able to answer your questions, please ask them anyway. It's very important that I capture all your questions and comments. When you've finished all the exercises, I'll answer any questions you still have."
- Describe in general terms what the participant will be doing.
Explain what all the materials are (such as the set of tasks, disks, and a questionnaire) and the sequence in which the participant will use them. Give the participant written instructions for the tasks.
If you need to demonstrate your product before the user observation
begins, be sure you don't demonstrate something you're trying to test.
For example, if you want to know whether users can figure out how to use certain tools, don't show them how to use the tools before the session. Don't demonstrate what you want to find out. For example, rather than showing the participant what the final design outcome looks like and asking an opinion, you can show before and after screen shots and ask what the participant observes about each one.
- Ask if there are any questions before you start; then begin
the observation.
- During the observation, remember several pointers:
- Stay alert. It's very easy to let your mind wander when you're in the seventh hour of running subjects. A great deal of the information you can obtain is subtle.
- Ask questions or prompt the participant. Make sure you have a
tester protocol that spells out how frequently you prompt and what
you say. Your interruptions shouldn't be frequent, but when a participant is hesitating or saying, "Hmmm," ask what the participant
is thinking about.
- Be patient; it is very easy to become impatient when someone is taking
a long time. The participant is doing you a favor and is probably somewhat nervous. Anything you can do to alleviate the participant's insecurities and put the participant at ease will provide you with much richer data.
- Conclude the observation.
Do the following when the test is over:
- Explain what you were trying to find out during the test.
- Answer any remaining questions the participant may have.
- Discuss any interesting behaviors you would like the participant
to explain.
- Ask the participants for suggestions on how to improve the product.
- Use the results.
As you observe, you will see users doing things you may never have expected them to do. When you see participants making mistakes, your first instinct may be to blame the mistakes on the participant's inexperience or lack of intelligence. This is the wrong focus to take. The purpose of observing users is to see what parts of your product might be difficult to use or ineffective. Therefore, if you see a participant struggling or making mistakes, you should attribute the difficulties to faulty product design, not to the participant.
Be sure to schedule time between your sessions to make notes and review the session. Jot down any significant points. If you used videotape or audio cassette tape, mark in your notes the specific parts of the tape that you may want to review.
To get the most out of your test results, review all your data carefully and thoroughly (your notes, the videotape or cassette tape, the tasks, and so on). Look for places where participants had trouble and see if you can determine how your product could be changed to alleviate the problems. Look for patterns in the participants' behavior that might tell you whether the product was understood correctly.
It's a good idea to keep a record of what you found out during the test. You don't need elaborate video equipment; a hand-held video camera will work. In fact, you don't even have to use video equipment. You can use a tape recorder to record what is spoken during the session. The important point is that you create some kind of objective, factual record of the session that you refer to later. That way, you'll have documentation to support your design decisions and you'll be able to see trends in users' behavior. You might want to write a report that documents the process you used and the results
you found. After you've examined the results and summarized the important findings, fix the problems you found and test the product again. By
testing your product more than once, you'll see how your changes affect users' performance.
|
